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Chapter I: The Measurement Big Picture 

This chapter is part of a practice guide designed to help state education agencies (SEAs) define 
measurement goals, select college and career readiness measures and indicators designed to 
support those goals, and use the data gathered with those measures and indicators to make 
informed decisions about college and career readiness policies, programs, and interventions. 
Chapter 1 focuses on setting the stage for the rest of the guide by defining the measurement 
landscape and exploring the SEA role in measurement goals related to college and career 
readiness and success. 


Throughout this practice guide, we will use a range of terminology to refer to measures. Although measurement 
terms are often used for different purposes or even interchangeably in texts, for the purposes of this guide, we 
will use the following definitions: 
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Measures refer to the full range of 
metrics designed to evaluate student 
and programmatic progress toward 
and achievement of college and 
career readiness and success. 

There are three categories of measures: 

• Indicators are measures that can be 
used to determine progress toward a 
milestone or established threshold. 
Students who perform at or above the 
threshold are more likely to be prepared 
for their college and career pursuits. 

■ Predictors are measures that are 
strongly correlated with improved 
postsecondary outcomes but for 
which a numeric threshold has not 
been established. 

■ Outcomes are measures that provide 
information on or a snapshot of student 
status after high school graduation. 
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Introduction 


Ensuring that students are college and career ready is increasingly a priority for states as a 
changing economic landscape has necessitated a shift in the goal of education from high school 
graduation to postsecondary success. As a result, stakeholders across the education landscape 
are exploring on-track indicators for and outcome measures of postsecondary success in an effort 
to incorporate them into the vast array of metrics already used to evaluate institutional, programmatic, 
and student progress and outcomes as well as educator effectiveness. The ultimate measure of 
college and career readiness is success (i.e. , whether students obtain a degree, an industry- 
recognized certification, and, eventually, a career that affords opportunity for advancement and 
provides a salary sufficient to support a family). But to effectively prepare students for success 
while they are in the prekindergarten to workforce (P-20W) pipeline, it is essential that SEAs, 
districts, and schools develop and select research-based measures that help detect and predict 
whether students who are moving through the pipeline are on track for success. Rigorous measures 
are those that allow SEAs and other state and postsecondary stakeholders to make labor market 
projections and analyze pending workforce gaps; evaluate the extent to which schools, programs, 
and teachers are effective at promoting college and career readiness for all students; and provide 
timely information, interventions, and supports to students who are not on track. 

Although college and career readiness measurement goals vary from state to state and across 
stakeholder groups due to differences in priorities, policies, and data systems, there are three 
overarching goals that drive these measures. College and career readiness measures are used to: 

1. Gauge student progress toward postsecondary preparedness 

2. Evaluate teacher and leader effectiveness in preparing students for success 

3. Evaluate the impact of policy, initiative, and program performance to improve SEA, 
district, and school effectiveness at preparing students for postsecondary success 

The roles that SEAs play in and the degree of influence they have over each of the overarching 
measurement goals vary. Although policy and program evaluation are emphasized in many SEAs, 
SEA involvement in the other measurement goals, which are primarily the purview of districts 
and schools, may be more limited. Across these three overarching measurement goals, however, 
the SEA can play an important role in providing guidance on the selection of measures and data 
use and in identifying appropriate data that can be used to meet multiple goals. Similarly, SEAs 
also can contribute to college and career readiness measurement efforts by aligning policies 
and providing measures directly to districts and schools, minimizing local burden. 

In the chapter that follows, we explore each of these overarching measurement goals in detail. We 
start with potential SEA roles in achieving each purpose and then explore potential uses for 
measures that may be leveraged in service of each goal. Finally, at the end of the chapter, we provide 
guiding questions to help SEAs identify who bears primary responsibility for each measurement 
purpose at the SEA, district, and school levels; define the current SEA role in each purpose; and, 
where possible, align measurement tasks to reduce duplication and burden on districts and schools. 
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Measurement Purposes 


STUDENT-LEVEL INDICATORS AND PREDICTORS 

At the fundamental level, college and career readiness measures must be used to provide 
information about the college and career readiness status of students who are currently in the 
P-20W pipeline. Student-level indicators are essential to determining whether students are on 
track for postsecondary success so that teachers and schools can provide needed support and, 
when necessary, intervene to reverse off-track trends. These indicators include a vast array of 
measures, such as early childhood school readiness metrics, formative and benchmark assessment 
scores disaggregated by skill or standards, early warning indicators for students who are off-track, 1 
and other diagnostic indicators incorporated into systems of academic or behavioral supports 
designed to help identified students get back on track. Strong evidence-based indicators and 
predictors of student preparedness act as a foundation for all other measures as they can 
provide insight into whether students, teachers, and schools are on track to meet their goals 
for student outcomes, such as college enrollment and persistence, degree or industry certificate 
attainment, and/or entrance into a middle- or high-skills career. 

Key to student-level indicators is that they are validated as predictive of college and career readiness 
and success. In addition, these indicators are valuable only if they trigger action by educators to 
support struggling students to get them back on track for college and career readiness and success. 
Frequent and systemic examination of indicators will allow educators to monitor student progress, 
identify students in need of support early, and determine interventions and supports that will be 
most effective in helping students meet their goals. 

Although tracking individual student progress is typically a school and district responsibility due 
to their ability to more immediately intervene, SEAs play an important role in helping schools and 
districts develop and use student-level indicators. SEAs have access to a broader range of data 
collected from schools and districts across the state and may use these data to provide indicators 
directly to schools and districts along with nationally validated indicators. SEAs also can and 
sometimes do establish statewide data systems in which districts and schools can house these 
indicators. Data systems, which may include statewide early warning systems or data dashboards, 
can help facilitate data use while providing potential indicators and thresholds for use across the 
state. These systems can serve as important resources for districts and schools, particularly if 
they integrate a wide range of measures that account for the diversity in geographic regions and 
populations and postsecondary goals that characterize many states. All SEAs can contribute to 
the effective use of student-level indicators by: 

► Validating indicators and thresholds that can be used as effective predictors 
of postsecondary success in the state context 


1 Originally, early warning systems were primarily used to identify students who were at risk of dropping out of high school. 
Increasingly, indicators of college and career readiness are being integrated into these systems to ensure 
that students are on track, not only to graduate from high school but also for future postsecondary success. 
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► Developing data systems, designed to produce easy-to-use, actionable reports, 
in which to house comprehensive and aligned indicator data 

► Providing guidance on integrating college and career readiness measures into 
existing data systems 

► Working across divisions and with other state agencies to develop data-sharing 
agreements and provide access to data on individual student performance and 
outcomes, in accordance with student privacy laws 

► Providing data to districts and schools in a usable and actionable format 

► Providing guidance and support on data use specific to individual student-level data 

The SEA role is essential to this measurement goal because, at the district and school levels, 
indicators of student progress are often collected but not always put to effective use. Without 
linking data that the indicators identify to actions that support student growth, even the strongest 
predictors are just a label. SEAs can play an important role in ensuring that districts and schools 
not only identify and implement valid indicators, but also work toward the ultimate goal of helping 
students stay on track for postsecondary success. 

Goals for the use of individual student-level indicators include: 

► Identifying students who are offtrack and providing interventions matched to data-based need 

► Identifying trends in aggregate or subgroup performance 

► Evaluating the effectiveness of strategies and interventions by examining enrolled student 
progress on these indicators over time 

Student-level indicators are not meant to: 

► Serve as one-time or summative assessments of student capabilities. Indicators track 
progress toward an outcome, rather than the outcome itself. 

► Transfer directly into school, district, or SEA impact or accountability measures. Indicators 
of student progress may or may not be indicators of school performance. 

► Measure teacher or program effectiveness as part of a formal or summative evaluation. 
These are predictive measures that may not have a one-to-one correlation with outcomes. 

As a result, they should not be used to make high-stakes decisions. 
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Student Progress Indicators in the States 

SEAs are increasingly investing in data systems that track and communicate student-level indicators across 
districts. Among the most common approaches are statewide individualized learning plans (ILPs) 2 implemented at 
the school level to help students plan their high school trajectory and evaluate whether they are staying on track. 

Forty-two states currently mandate the use of statewide ILPs either for subsets of schools in need of improvement 
or for all schools across the state. For example, in 2013, the Colorado State Board of Education passed new rules 
governing the state’s use of ILPs or what they call the Individual Career and Academic Plan (ICAP) . These rules 
ensure that each public school student in the state has a personalized plan to help him or her set goals for 
postsecondary success and track progress toward those goals. Plans are developed by students and discussed 
regularly with parents or guardians, school counselors, administrators, teachers, and/or other school personnel 
to ensure that each stakeholder has an understanding of the student's desired postsecondary outcomes, the 
pathways he or she will take to get there, and the benchmarks that he or she must achieve along the way to 
demonstrate mastery within the pathway. The plans include: 

■ A postsecondary and workforce goal 

■ Yearly benchmarks that demonstrate meeting that goal 

■ Academic progress, including dual credit or credit recovery/ remediation 

■ Assessment scores, broken down by postsecondary readiness skills 

■ Documentation of contextual and/or service learning experiences 

■ Progress toward applying to college, securing scholarships and loans, and developing an understanding of 
the financial impact of postsecondary education or training (Colorado State Board of Education, 2013) 

The ICAP demonstrates a way in which SEAs can provide guidance on which student-level indicators to use. The 
Colorado State Board of Education and many SEAs are still exploring best practices for aligning these indicators 
with other data systems in their states and for supporting districts and schools as they explore how to use these 
indicators to ensure that students are on track for postsecondary success. 


TEACHER AND LEADER EVALUATION MEASURES 


Measures included in teacher and leader evaluations can be used to provide school personnel 
with data to improve instructional practice as well as allow administrators and district staff to 
make informed decisions about human capital. College and career readiness teacher evaluation 
measures may take one of three forms. First, observable performance indicators, such as whether 
the teacher spends time discussing college options or linking daily objectives to real-world 
applications or career pathways in which they could be used, are often captured by observations 
conducted by administrators or other personnel. Aggregate student performance indicators, which 


2 ILPs are dynamic, personalized, student-driven educational “roadmaps” that are used to record and monitor students’ 
academic, career, and personal ambitions and accomplishments (Rennie Center for Education Research and Policy, 2011) 
Although ILPs evolved from individualized education programs (lEPs), ILPs are most commonly used to track learning 
goals and pathways for all students, while lEPs are used specifically for students with disabilities as mandated by the 
Individuals with Disabilities Education Improvement Act of 2004. 
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may include grade point average, state assessment scores, or scores on other exams, such as 
the SAT, ACT, Af?or IB tests, are often incorporated into summative evaluations. Finally, student 
growth indicators based on state assessments administered in tested grades and subjects 
(i.e., in mathematics and reading/language arts in Grades 3-8 and once in high school) are 
increasingly incorporated into teacher evaluations across districts and SEAs due to federal 
and state policy requirements. 

Although student performance and growth metrics are increasingly incorporated into teacher 
evaluation systems across the country, more research is needed to help develop best practices 
for linking these measures to secondary school teacher performance, particularly in nontested 
subjects. Similarly, research on leader evaluation lags behind teacher metrics. Often, measures of 
school climate are used as proxies for administrator effectiveness. College and career readiness 
climate measures might include establishment of a college-going or achievement-oriented culture, 
student-counselor relationships, and the extent to which the school supports and cultivates 
students’ social and emotional needs (Center on Great Teachers and Leaders, 2014; Clifford, 
Menon, Gangi, Condon, & Hornung, 2012). 

Although teacher evaluation has historically been the purview of districts and schools, since 
the addition of the highly qualified teacher (HQT) provision in the 2001 reauthorization of the 
Elementary and Secondary Education Act (ESEA), as amended, and ESEA flexibility, 3 SEAs have 
increasingly played an important role. SEAs can help districts build effective teacher and leader 
evaluations that integrate measures of college and career readiness by: 

► Identifying college and career readiness measures to integrate through research 
and stakeholder convenings as part of a continuous improvement process 

► Providing guidance on teacher and leader evaluation requirements to clarify when formative 
and summative evaluations will be conducted, how often, and by whom to ensure feedback 
is relevant, timely, useful, and conducted by an evaluator with expertise in the college and 
career readiness relevant fields 

► As applicable, setting weights for college and career readiness measures and indicators 

► Providing timely, accurate, accessible, and actionable data to be integrated into student 
performance and growth indicators 

► Cultivating relationships with institutions of higher education to establish feedback loops 
whereby teacher evaluation data are used to inform teacher preparation program curricula 
and decision making specific to preparing students for postsecondary success 

Along with these roles, SEAs must bear in mind that in the current era of increased accountability, 
teacher and leader evaluation can often be a source of tension in districts and schools if the results 
are used punitively rather than informatively. These tensions are heightened by the fact that teacher 
impact on student progress is complex and necessarily influenced by other factors both within and 
outside of the school. In addition to providing guidance on how teacher and leader evaluation can be 
structured and implemented, SEAs also can communicate that, although these evaluations do serve 
as assessments of performance, it is important to remember that this is not their only or, indeed, 
primary role. Instead, the use of these evaluations to improve practice is the ultimate aim of 
these systems. 


3 In ESEA flexibility, these evaluation systems are referred to as “teacher and leader evaluation and support systems.” 
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Goals for use of teacher and leader evaluation measures include: 

► Informing teacher and leader practice and improvement 

► Making informed decisions for targeted professional development 

► Making personnel decisions about classrooms of best fit and equitable assignment 
of effective educators across classrooms in a school and schools within a district 

Evaluation measures are not meant to: 

► Serve as one-time or summative evaluations of teacher or leader capabilities. Evaluation 
is not an end in and of itself. Improvement should be the goal. 

► Establish a direct correlation between teacher input and student performance, isolated from 
all other mediators. Student performance is complex and linked to many factors both in and 
outside of school. 


Teacher Evaluation Measures in the States 

Under ESEA flexibility, many SEAs and districts are just beginning to explore how best to integrate college and 
career readiness measures into teacher and leader evaluation and support systems for all teachers. Although the 
HQT requirements under the 2001 reauthorization of the ESEA only required teacher certifications for those who 
taught core academic subjects, a number of SEAs and districts have worked since then to exceed HQT requirements 
by developing innovative measures to evaluate teachers of nontested subjects as well, such as career and technical 
education (CTE) courses. Common measures used to assess the effectiveness of CTE teachers include: 

■ Student mastery of CTE student learning objectives (SLOs) 

■ Performance on teacher-designed or state CTE exams 

■ Schoolwide student growth in tested grades and subjects 4 

■ Classroom observation 

■ Portfolios and/or classroom artifacts 

Twelve states have provided guidance on evaluating CTE teachers, primarily focused on which of the above measures 
can be used to capture student growth. Student growth is a particular challenge for CTE because student assessment 
often takes the form of an industry certification exam rather than technical skills assessments that demonstrate 
benchmarks of progress. Industry certification exams are generally not designed to measure growth over time, but are 
summative evaluations of student knowledge. Ten of the 12 states evaluate student growth by looking at SLOs, such 
as those found in the Common CareerTechnical Core or other state CTE standards, while four of the 12 states use 
state-, district-, or teacher-developed end-of-course assessments to determine student growth (Jacques & Potemski, 
2014; Garcia & Stephan, 2011). As SEAs work to integrate new evaluation measures specific to college and career 
readiness, CTE evaluation measures that gauge student progress using metrics beyond test scores provide useful 
examples of how new measures may be integrated into existing evaluation systems. 


4 For federal accountability purposes, these include mathematics and reading/language arts in Grades 3-8 and once 
in high school. 
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POLICY AND PROGRAM EVALUATION MEASURES 


Many state college and career readiness measurement goals center on evaluating policies, 
programs, and initiatives to make strategic decisions about policy shifts, intervention at the 
district and school levels, and resource allocation. Many measures can effectively inform these 
decisions, including program enrollment data, aggregate student progress measures, and outcome 
measures that demonstrate trends in learners’ postsecondary success. SEAs have multiple sets 
of measures that are used to inform these decisions; common formal measurement systems 
include accountability measures at the federal and state levels, statewide school and district 
report cards or statewide high school feedback reports, and statewide longitudinal data 
systems (SLDS). 


Policy and program evaluation measures are essential components of continuous improvement cycles that allow 
stakeholders at all levels of the education spectrum to make informed decisions about college and career 
readiness initiatives. Table 1 illustrates the ways in which these measures can be used. Although the purposes 
differ, there is often a larger degree of overlap across the data used to inform these purposes. 


Table 1. Policy and Program Evaluation Purposes 


Federal 

State 

District 

School 

Informing Policy Decisions 

X 

X 

X 


Allocating Resources 

X 

X 

X 


Selecting and Assigning Programs* 


X 

X 


Selecting and Assigning Student Interventions* 



X 

X 

Making Decisions About Existing Programs* 


X 

X 


Making Decisions About Existing Interventions* 



X 

X 


"■Programs refer to school- or institution-wide college and career readiness strategies, while interventions refer to strategies that may target 
individual students or student subgroups. 


Policy and program evaluation measures are important for stakeholders throughout the P-20W 
spectrum. For example, the federal government uses accountability measures to make decisions 
about resource allocation, while districts and schools might use similar information to target 
interventions for specific student subgroups. Institutions of higher education and workforce 
stakeholders partner with SEAs to establish feedback loops using SLDS, collaborating to ensure 
that learner needs are aligned with the workforce needs of the state. And parents and students 
can use report card data to make decisions about which school learners will attend based on 
their interests and aptitudes as well as the school’s performance. SEAs play a key role, not only 
in using these data to make informed decisions, but also in facilitating data sharing and use at 
other levels so that stakeholders across the system can do the same. 


COLLEGE AND CAREER READINESS AND SUCCESS CENTER 




SEAs often play multiple roles to support the data collection for and data-based decision 
making that stems from program evaluation measures. SEAs help stakeholders identify 
evaluation measures by: 

► Establishing metrics for federal accountability measures where flexibility is allowed 

► Developing state accountability systems 

► Establishing requirements for what must be reported for state accountability systems 
or statewide report cards 

► Collaborating with institutions of higher education, state departments of labor and other 
departments where workforce data is housed, and other national data sources to share 
postsecondary outcome measures 

► Aligning measures and/or providing guidance on which measures can be used for multiple 
purposes across levels and policies and initiatives to reduce burden on districts and schools 

SEAs help report on evaluation measures and support others to do the same by: 

► Collecting data to comply with federal and state accountability provisions 

► Providing guidance to districts and schools to facilitate data collection and compliance 
with accountability provisions at both the federal and state levels 

► Making data available in easily usable forms for those who will analyze the data or 
otherwise use them to make decisions 

► Ensuring that report cards are easily accessible on SEA websites and encouraging or 
requiring districts to do the same 

► Providing guidance on federal and state policy concerning privacy, data sharing, and data use 

SEAs make data-based decisions and support others to do the same by: 

► Evaluating program and institutional effectiveness to make decisions about program 
continuation and resource allocation 

► Evaluating policies to make decisions about legislative shifts, continuity, or new policy needs 

► Providing guidance on and building district capacity to make data-based decisions 

► Providing guidance on interventions that align with data-based needs 

Because policy and program evaluation measures are ultimately designed to facilitate data 
sharing and decision making, it is essential that these measures are selected with an eye toward 
data use. This process includes both selecting measures that are appropriate for the evaluative 
end as well as ensuring that these data are accessible and communicate actionable information 
to appropriate stakeholders (Data Quality Campaign, 2013). Evaluation measures not only inform 
college and career readiness strategies and initiatives, but also are a mechanism through which 
SEAs, districts, and schools are held accountable to parents, students, the U.S. Department of 
Education, and other stakeholders. 

Goals for use of policy and program evaluation measures include: 

► Making decisions about allocation of resources, including funding, staffing, educator 
professional development, and other programmatic interventions and supports 

► Identifying districts and schools that need to improve and helping these institutions 
target areas for improvement 
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► Helping parents make decisions about the districts and schools in which to enroll their children 

► Facilitating data sharing from the K-12 system to other stakeholders, such as institutions of 
higher education and the workforce 

► Demonstrating program effectiveness and/or student need to build political will 

Policy and program evaluation measures are not meant to: 

► Exclusively fulfill requirements with no other analysis or use 

► Determine individual student performance. Policy and program evaluation measures will examine 
aggregate student results and should not be used to make inferences about individuals. 

► Unduly influence decisions about programs or policies if the measures do not yet have a well- 
established research base 

► Unduly influence decisions about programs or policies if the measures do not yet have a 
research-based link to school interventions or impact 


Policy and Program Evaluation Measures in the States 

Many SEAs make potential policy and program evaluation measures publicly available through statewide school and 
district report cards or high school feedback reports.These report cards often include federal or state accountability 
metrics as well as information about programming, school climate, and other assessment measures. The Kentucky 
School Report Card, which touts a tagline of “College/Career Readiness for All,” is designed to share school 
performance data with students and their families (Kentucky Department of Education, 2014) and includes both 
state-level college and career readiness accountability measures and programmatic measures designed to help 
parents, students, and other stakeholders make data-informed decisions. Accountability measures include indicators 
of college and career readiness such as: 

■ Receiving a high school diploma 

■ Meeting benchmark levels on the ACT 

■ Performance on college placement tests 

■ Meeting benchmark levels on career-ready academic tests 

■ Meeting benchmark levels on career-ready technical tests or obtaining an industry certification 

Because most students take either the ACT and college placement tests or a career readiness test, such as the 
Armed Services Vocational Aptitude Battery (ASVAB), the Kentucky Department of Education has worked to 
establish comparable cut scores that would demonstrate readiness for both college and careers. Scores on 
ASVAB subtests, such as Paragraph Comprehension, Word Knowledge, Math Knowledge, and Academic Reasoning, 
have been identified as predictors of ACT scores in reading, English, and mathematics, enabling the Kentucky 
Department of Education to establish valid and comparable cut scores for both measures within its state 
accountability system. 

In addition to these accountability measures, Kentucky includes both programmatic and performance measures 
relevant to college and career readiness on its report cards. Report cards include information on whether dual 
enrollment and National Academy Foundation courses are offered, students enrolled and earning a certificate in 
career pathways, student performance on end-of-course assessments, and outcome measures of success, such 
as college attendance, vocational training, working, and enlistment in the military. 
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Guiding Questions for Chapter 1 


Guiding Question 


Measurement 

Purposes 

Who is responsible for 
each measurement 
purpose? 


SEA Role 

How does the SEA 
engage with each 
measurement 
purpose? 


Further Prompts 


For each measurement purpose detailed above, ask the following questions: 

■ What questions do you want your data to help answer? 

■ Who decides which measures of college and career readiness currently are 
selected, collected, and used to help answer these questions? 

■ What systems are used to house these measures? 

■ How are these measures currently being used? For what purpose? By whom? 

■ What decisions do the data collected with these measures inform? 

■ What level of impact (i.e., student-level, school-level, district-level) do you 
hope to achieve through use of these data? 

■ Who is the primary audience for these data? With whom will these data 
be shared? 

■ What gaps exist that might be informed by data from these measures? 
What new data would help inform our measurement purposes? 

■ What further information do you need about these measures and/or data? 
How could you find out? 

For each measurement purpose detailed above, ask the following questions: 

■ What is the SEA’s role in accomplishing the measurement purpose? 

■ What is the SEA’s role in data use associated with each measure? 

■ Who at the SEA is involved with work related to each type of measure and 
associated data use? Who else should be involved? 

■ Who outside of the SEA do you work with concerning each type of measure 
and associated data use? Who else should be involved? 

■ How does the SEA keep track of the initiatives and activities related to 
these measures at the district and/or school levels? 

■ How does your agency communicate with districts and schools about each 
measurement purpose? 

■ What are the SEA’s expectations for districts’ and schools’ use of 
these measures? 

■ How does the SEA provide data to districts and schools for various 
measurement purposes? What considerations are taken into account? 

■ What message is conveyed by related guidance and training materials 
about use of these measures? 

■ Where are the gaps in what the SEA provides? What else is necessary to help 
districts and schools or your SEA accomplish each measurement purpose? 
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■ Are there other college and career readiness initiatives or measures 
currently being implemented at the SEA level? 


Measurement 

Alignment 

Have the measurement 
purposes been aligned 
to the SEA’s strategic 
plan and/or other SEA 
measures of college 
and career readiness? 


How will the adoption of college and career readiness measures affect 
other measurement initiatives? 

How can measures be aligned to create a coherent system? What are the 
barriers to ensuring there is measurement alignment? How will the SEA 
address these barriers? 

Is there flexibility for district in put/ integration of existing district measures 
to facilitate greater alignment? 

How can measures be aligned with other systems (i.e., early learning, 
institutions of higher education, U.S. Department of Labor)? 


■ What measurement lessons can be learned from districts and schools that 
could replicated at the SEA level? 
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